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ANALYSIS & SOLUTIONS 


Executive Summary 


The Every Student Succeeds Act (ESSA) provides greater flexibility in state 
accountability systems than did previous federal legislation. In response, many states 
continue to refine their accountability systems to include college readiness tests, 
including college admissions and placement exams. This paper summarizes 
perspectives of K-12 educators, postsecondary educators, and researchers on these 
measures to inform policymakers as they revise accountability systems under ESSA. 


CNA conducted a literature review on college admissions and placement exams, 
focusing on their uses in high schools and in K-12 accountability systems. The review 
included a policy scan of state plans under ESSA, as well as research on college 
admissions and placement exams in accountability settings. 


Dozens of college readiness exams have been developed at the national, state, and 
institution levels. College admissions exams assess aptitude and likelihood of 
postsecondary success, whereas placement exams assess subject knowledge to 
determine preparedness for specific college courses. The exams vary considerably in 
their content, rigor, and ability to predict postsecondary success. 


Research suggests that admissions exams show greater rigor and closer alignment 
with K-12 standards than do placement exams (Achieve, 2007). Several researchers 
questioned the predictive validity of both exam types, finding that they often led to 
placement errors in college courses. A combination of high school GPA and exam 
results was found to be more comprehensive than exam results alone (Belfield & 
Crosta, 2012; Scott-Clayton, Crosta, & Belfield, 2014; Scott-Clayton & Stacey, 2015). 


Some authors argued that such assessments—especially those that are nationally 
known—can raise awareness of postsecondary options, boost confidence in state 
accountability systems, and provide students with admissions data that are accepted 
at many colleges (Miller & Happel, 2011). Others question the suitability of readiness 
exams as accountability measures given the testing burden and the exams’ limited 
alignment both with state standards and with accountability measurement purposes 
(Martineau & Marion, 2015; Catchpole, 2016). 


Neither policymakers nor researchers agree on whether or how to use college 
readiness exams to measure state accountability. In 2016, 23 states included at least 
one such exam (ACT, SAT, ACCUPLACER, or COMPASS) in their accountability 
systems. 
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Recommendations for K-12 educators based on the research and policy review 
include the following: 


Match the exam’s purpose to its use. 
Use multiple readiness measures. 


Customize admissions tests when incorporating them into statewide testing 
systems. 


Modify high school tests to measure college readiness. 


Provide support for districts and schools to assist students who do not meet 
benchmarks. 
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Introduction 


The Every Student Succeeds Act (ESSA), signed in December 2015, provides greater 
local control over accountability and school improvement efforts. It ends the federal 
requirements for Adequate Yearly Progress and permits each state to design its own 
accountability system to meet minimum federal requirements, such as ensuring that 
measures include all students and student subgroups. ESSA requires five indicators 
for state accountability systems: proficiency on assessments, growth in proficiency in 
grades below high school, high school graduation rates, progress of English learner 
students toward English language proficiency, and a state-selected indicator of 
school quality or student success (Aragon, Griffith, Wixom, Woods, & Workman, 
2016). States have developed accountability plans under ESSA but continue to need 
support making informed decisions as they craft long-term accountability systems. 
This paper provides additional information about elements of college readiness as 
part of a comprehensive state accountability system. 


Currently, half of all U.S. students entering college take remedial courses, averaging 
2.6 remedial courses per student (Scott-Clayton & Stacey, 2015). To address a lack of 
student postsecondary readiness, 30 states have designed accountability systems 
that include at least one measure of college and career readiness, including exams or 
coursework, but states vary considerably in the specific measures used and in the 
way performance on these measures is assessed (Martin, Sargrad, & Batel, 2016). 
Many of these states use college admissions exams, college placement exams, or both 
in their accountability systems to measure readiness. 


e Admissions exams are used to determine eligibility for enrollment at 
postsecondary institutions. These exams include the SAT and the ACT, which 
aim to measure academic aptitude or predict postsecondary success. 


e Placement exams are used to determine which courses students are prepared 
to take in postsecondary institutions. These exams include ACCUPLACER and 
COMPASS, which aim to diagnose current academic knowledge (Achieve, 2007). 


Colleges and universities have long used these exams for admissions decisions, 
course placement, and awarding of merit-based scholarships. But now these exams 
are taking on new roles. States use a combination of these exams for high school 
accountability purposes and to inform high school instructional decisions, though 
the exams were not designed for these purposes (Achieve, 2007; Southern Regional 
Education Board [SREB], 2014). Further, colleges and universities are slowly but 
increasingly moving away from admissions exams as selection criteria, while 
emphasizing placement exams to assign students to coursework (Belasco, Rosinger, 
& Hearn, 2015). 
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As state departments of education revise assessment and accountability systems 
under ESSA, they would benefit from a better understanding of how K-12 educators, 
postsecondary educators, and researchers view the use of these instruments. This 
paper provides some of this information. First, we describe the methods we used and 
the study limitations. Next, we briefly discuss the common admissions and 
placement exams in use today across the country and issues to consider in their 
inclusion in accountability systems. We conclude with recommendations for state 
departments of education. 


What we did 


We conducted a literature review to identify information about uses of college 
admissions and placement exams, especially in high schools and state K-12 
accountability systems. This review was comprised of two parts: a policy scan of 
state ESSA plans and a research review using EbscoHost databases. 


The first activity was a document analysis of state accountability plans to examine 
how states currently identify and measure progress toward college and career 
readiness, with an emphasis on college readiness assessments. Accountability plans 
for all states under No Child Left Behind are _ publicly available at 
https://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html. 


We supplemented this information with ESSA state plan submissions that were 
available at the time of writing. This process allowed us to identify the common 
assessments, as well as all alternatives used across the country. 


The second activity was a literature review on college admissions and placement 
exams. This review helped us identify advantages and disadvantages of such exams, 
as well as potential unintended consequences. 


Limitations 

This study examines current literature about the uses of college readiness 
assessments. It does not examine the characteristics of individual exams—such as 
validity, reliability, or specific psychometric properties—to determine rigor or 
comparability. Such an analysis is beyond the scope of this study. In addition, this 


study does not attempt to identify all, or a representative sample of, admissions and 
placement exams that are in use at K-12 or postsecondary levels. 


NA 
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Findings 


Research on uses of admissions and placement exams in high school settings or for 
high school accountability is limited. Most of the literature that we identified focused 
on college uses of admissions and placement exams—the historical and intended 
use. Nevertheless, K-12 educators can draw some lessons from the data. 


The fourcommon college readiness exams 


At the postsecondary level, there are national admissions exams, national placement 
exams, state- or system-wide exams, and institution-level exams. Many college 
readiness exams are used nationally. Exams include the following examples: 


e National admissions exams: the ACT and the SAT. 


e National placement exams: COMPASS (by ACT, Inc.) and ACCUPLACER (by the 
College Board). 


e State- or system-wide placement exams: Texas Higher Education Assessment, 
Washington Intermediate Mathematics Placement Test, California State 
University Early Assessment Program—Direct Writing, Kentucky Online Testing 
(KYOTE) College Algebra. 


e Institution-level placement exams: Mathematical Association of America 
Algebra Placement Test, University of Minnesota Calculus Readiness Test, 
Youngstown State University Composition Placement Test (Achieve, 2007). 


These assessments vary widely in content. Some cover multiple subjects—the ACT 
includes math, reading, English, and science sections. Others, such as the University 
of Minnesota Calculus Readiness Test, target a single subject or subtopic. 


The National Assessment of Educational Progress (NAEP) conducted a survey in 2012 
to understand which exams colleges use to determine remediation needs and the 
requisite scores on those exams. (The study did not look at use of exams for college 
admission.) NAEP estimated that 71 percent of colleges (including both two- and 
four-year institutions) used at least one assessment to place students in mathematics 
courses, and 53 percent used a reading exam. The most common exams used for 
placement were the ACT, the SAT, ACCUPLACER, and COMPASS (Table 1), and the 
benchmarks required on the exams varied by institution (Fields & Parsad, 2012). 


Table 1. Percentage of colleges that used specific exams for course placement, 


2012 
Mathematics Reading 

ACT 23% COMPASS Reading 22% 
A . 

COMPASS Algebra 20% assis ham aiinads 19% 
Comprehension 

SAT 17% | ACT 16% 

A PLAC ER El 

hecie: ea pens 16% | SAT 11% 

ACCUPLACER College-Level 

onsite ? 5% | ASSETReading Skills 9% 

COMPASS College Algebra 4% Other 10% 

Other 22% 


Source: Fields & Parsad (2012). 


Note: Some collegesuse multiple exams; therefore, valuesdo not sum to the total 
percentage of colleges that use at least one exam described in the text. 


Growing use of college readiness 
assessments in K-12 


At the K-12 level, policymakers increasingly are using performance on college 
admissions and placement exams as indicators in state accountability systems. As of 
2016, 23 states used performance on exams such as the ACT, the SAT, ACCUPLACER, 
or COMPASS as an indicator in their accountability systems (Martin, Sargrad, & Batel, 
2016). Such exams help develop a system of multiple measures that can indicate 
progress toward college readiness (English, Rasmussen, Cushing, & Therriault, 2016). 
Other states administer readiness exams but do not require them for 
accountability—such as Tennessee, which requires the ACT for all grade 11 students 
for diagnostic purposes but not for accountability. 


Some studies have looked at impacts of high school-based college readiness testing. 
In California, the Early Assessment Program includes a voluntary, state-developed 
college readiness assessment in grade 11. This assessment reduced the need for 
remedial education in college math and English by 2 to 3 percentage points, with the 
largest impact for students near the readiness cutoff (Kurlaender, Jackson, Grodsky, 
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& Howell, 2016). In Maine, the requirement for all grade 11 students to take the SAT 
raised college-going rates by 2 to 4 percent overall and by 10 percent for students 
who would not have otherwise taken the SAT (Hurwitz, Smith, Niu, & Howell, 2015). 


Align an assessment’s purpose and use 


An assessment’s purpose should align with its use (Morgan & Michaelides, 2005). 
Even if an exam is internally valid and reliable, educators must use the results in the 
right way (Scott-Clayton, 2012). 


Educators may want to ask: 


e What is the assessment’s purpose? Is it diagnostic? Predictive of specific 
outcomes? 


e What decisions can be made using the exam results? 
e What decisions do educators want to make? 


e Who is using the data to make decisions—high school teachers, college 
counselors, state agencies, students, or someone else? 


For example, colleges use the ACCUPLACER placement exam in a variety of ways and 
do not always use all the data or tools embedded in the assessment system (College 
Board, 2012). Relevant information includes high school GPA, weights for student 
backgrounds that schools can adjust, diagnostic assessments, online intervention 
tools, and modularized placement tests. “How, when, and why ACCUPLACER is used 
will impact the outcomes achieved” (College Board, 2012). 
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Using the ACT in achievement contexts: Wyoming 


In 2017, the Wyoming State Board of Education adopted or extended a set of 
college and career readiness assessments as part of its accountability system. All 
grade 9 and 10 students take ACT’s ASPIRE, grade 11 students take the ACT Plus 
Writing, and grade 11 and 12 students have the option to take the WorkKeys exam 
(Wyoming State Board of Education, 2017). These decisions were based on state 
legislative guidance to provide college and career readiness assessments earlier in 
high school and on similar recommendations from the Assessment Task Force. 


Wyoming had planned to use the ACT as the grade 11 achievement indicator in the 
state’s accountability system beginning with the 2017/18 school year (Wyoming 
Department of Education, 2016). However, in early 2017, the U.S. Department of 
Education informed the state that it would need to provide substantial evidence to 
support this use of the assessment for achievement (Gewertz, 2017). According to 
the submitted ESSA plan, the state is still planning to use this achievement 
measure (Wyoming Department of Education, 2017). 


Wyoming policymakers discussed the appropriate use of college readiness exams 
in the accountability system extensively. The Wyoming Assessment Task Force 
recommended using the ACT or SAT only as a postsecondary readiness 
indicator, rather than as an achievement indicator in the state accountability 
system. The reasons for this recommendation included the lack of validity of 
these assessments as an achievement measure, the lack of alignment between the 
assessments and Wyoming’s content standards, and the lack of instructionally 
actionable information for educators from ACT score reports. The task force also 
recommended that these assessments be implemented in a comprehensive 
accountability system using multiple measures (Martineau & Marion, 2015; 
Catchpole, 2016). 


Such issues should be weighed by other states as well as they consider similar 
actions for their accountability systems. 


Admissions exams appear to be more 
rigorous than placement exams 


Achieve analyzed individual items from college admissions and placement exams to 
compare them with each other and with college and career readiness benchmarks 
created by the American Diploma Project (ADP). The exams included the ACT, the 
SAT, statewide tests, and institution-specific tests (Achieve, 2007). 


Achieve (2007) found that expectations for college curriculum were consistent 
among college faculty across states and institutions: 


e Math: knowledge and skills from a four-year mathematics sequence including 
Algebra I and II, Geometry, data analysis, statistics, mathematical reasoning, 
and problem solving. 


e English: skills to communicate effectively to different audiences and analyze 
various types of texts. 


However, the study concluded that admissions and placement exams vary 
considerably in their content and rigor and that they do not fully measure the 
knowledge and skills that are included in the ADP benchmarks. Admissions exams 
were more demanding than placement exams and better balanced in the types of 
questions asked. This study is now a decade old, however, so the results may be 
outdated. 


Admissions and placement exams have 
limited predictive validity 

Research suggests that admissions and placement exams have limited ability to 
predict student success in college. However, authors caution that many factors 


contribute to this finding, such as exam validity limitations, nonacademic factors not 
captured by the exams, and the way administrators use exam data (Fulton, 2012). 


Course placement errors 


Research from the Community College Research Center (CCRC) examines 
overplacement and underplacement for the COMPASS and ACCUPLACER placement 
exams from the ACT and the College Board, respectively.! Overplacement occurs 


‘In 2015, ACT Inc., informed by research on the predictive value of the COMPASS, decided to 
phase out the assessment as a product line (Fain, 2015). 
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when students who are not predicted to succeed in college-level courses are placed 
into those courses anyway, and underplacement occurs when students who could 
succeed in college-level courses are placed into remedial courses. 


According to several studies, both COMPASS and ACCUPLACER lead to significant 
misplacement in courses, and GPA is a better predictor of success. Underplacements 
into remediation were more common than overplacements. Underplacement rates 
varied by study and community college location, with underplacement in an urban 
community college as high as 33 percent (Scott-Clayton & Stacey, 2015). 


The CCRC also calculates a “severe error rate,” which combines students predicted to 
earn a B or better in a college-level course (that is, to be highly successful) but who 
are placed into remediation with students placed into a college-level course but 
predicted to fail there, based on regression analyses (Scott-Clayton, 2012). The severe 
error rates for placement were between 27 and 33 percent in English and between 21 
and 28 percent for math, using either COMPASS or ACCUPLACER (Belfield & Crosta, 
2012). Adding GPA, or using only GPA, to make placement decisions reduced both 
overplacement and underplacement by up to 30 percent (Belfield & Crosta, 2012; 
Scott-Clayton, Crosta, & Belfield, 2014). In fact, a pilot at Long Beach City College that 
used GPA or other high school information for placements saw the percentage of 
students who passed college-level English triple (Scott-Clayton & Stacey, 2015). 


Predicting othercollege outcomes 


Studies have also examined how admissions and placement exams predict other 
college outcomes, such as GPA, specific course grades, and dropout rates. The results 
are mixed. 


College GPA. For the ACT, one study found that only the math and English 
components predict college GPA in freshman and sophomore years (Bettinger, Evans, 
& Pope, 2013). The ACT reading and science components were not predictive of 
success (individually, in combination with math and English, or through the 
composite score). The study, conducted at four-year public universities in Ohio using 
data from 1999, was confirmed in 2006 using private college data and a smaller 
cohort. The results held for student subgroups as well. The study concludes by 
stating that a composite math-English ACT score has better predictive power than 
the ACT composite score of all four subtests. 


COMPASS and ACCUPLACER were weakly associated with college GPA, though the 
association was not present when including high school GPA (Belfield & Crosta, 2012; 
Hodara & Cox, 2016). Even a study by ACT Inc., which produces COMPASS, agreed 
that using high school GPA in combination with COMPASS is better than either on 
their own (Westrick, 2014). 


Course completion. COMPASS math was found to be a better predictor than 
COMPASS English for passing a college-level course (Scott-Clayton, 2012). Also, 
COMPASS more accurately predicts earning a B in a college course than earning a C. 
That is, it is easier to predict who will do very well in a course than it is to predict 
who will be borderline or may fail (Scott-Clayton, 2012). 


Credits earned. ACCUPLACER results were associated with college credits earned, 
even after controlling for high school GPA. Specifically, students with the highest 
quartile of ACCUPLACER scores earned 9 credits more than students in the lowest 
quartile after three to five semesters (Belfield & Crosta, 2012). 


Dropout rate. As above for college GPA, one study found that the math and English 
components of the ACT predict dropout rate in the first and third years. However, 
the reading and science components did not accurately predict dropout (Bettinger, 
Evans, & Pope, 2013). 


Exam equivalency 


One consideration is exam equivalency. Ellis (n.d.) calculated the following college 
readiness scores in West Virginia for the ACT, the SAT, ACCUPLACER, COMPASS, 
ASSET by ACT, and the Test for Adult Basic Education (TABE) (Table 2). Other states 
or colleges may use their own values, so this table is a guide to begin discussion 
rather than an authoritative conclusion. 


Table 2. Calculated college readiness scores for several admissions and 
placement exams 


ACT SAT ACCUPLACER COMPASS ASSET TABE 


Hace cc acc 


19 103 (Arithmetic ) 58-61 (Pre-Alg.) | 44-45 (Numerical) 12.7-12.9 
97 (Elem. Alg.) 


Source: Ellis (n.d.) 
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Are college readiness exams necessary in 
high schools? 


Some educators and researchers question the necessity of using college admissions 
or placement exams in high school accountability systems at all. Common arguments 
against using college readiness exams include the following: 


e They do not measure state academic standards. 


e Norm-referenced exams, such as admissions or placement exams, do not show 
student improvement over time. 


e Domain-sampled tests do not show mastery of content. 
e High stakes put too much pressure on students 


e There are potential testing biases based on race, ethnicity, and socioeconomic 
status (Buchmann et al., 2011; Miller & Happel, 2011). 


Those suggesting that the exams are not necessary also point out that more than 50 
selective liberal arts colleges no longer require an admissions exam, with 800 
postsecondary institutions nationwide adopting test-optional admissions policies 
(Belasco, Rosinger, & Hearn, 2015). These colleges question whether admissions or 
placement exams provide additional actionable data beyond GPA (see above) and 
hope to increase diversity by eliminating potential testing biases against race or 
socioeconomic status (SES). The research supporting an impact on student diversity 
is mixed, however. After controlling for many factors, the number of applications at 
test-optional colleges rose, along with reported test scores (perhaps because students 
with low exam scores did not report them), but the diversity of enrolled students did 
not increase. In fact, test-optional policies may institutionalize enrollment disparities 
by increasing admissions committees’ reliance on characteristics that are slanted 
toward white, high-SES students and schools (such as Advanced Placement access or 
extracurricular activities) (Belasco, Rosinger, & Hearn, 2015). 


Some argue for including admissions exams in accountability systems. They suggest 
that the benefits of using admissions exams in K-12 education include the following: 


e Confidence in the system based on use of well-known exams with strong 
connections to education. 


e Increased awareness among students about the importance of going to college. 


e Increased awareness among students and educators about college readiness to 
inform instructional decisions before applying to or enrolling in college. 


e Wider acceptance by colleges of national exams than of local exams. 


e Potential college opportunities for students who otherwise may not have 
applied (Miller & Happel, 2011). 
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Recommendations from the literature 


The research provides several recommendations for K-12 educators in both planning 
and implementing college readiness exams as part of accountability systems: 


Match the exam’s purpose to its use. No exam is valid for all uses. Review the 
available validity evidence to compare with the intended use and interpretation 
of scores (Miller & Happel, 2011). Assessment and curriculum need to be 
linked across high school, exam, and college (Hodara, Smith Jaggers, & Mechur 
Karp, 2012). If the exam results inform high school college readiness 
interventions, be sure the exam is relevant to the subsequent course (for 
example, introductory algebra, intermediate algebra, or basic statistics) (Scott- 
Clayton, 2012). 


Use multiple measures. No single measure is likely to capture the multiple 
dimensions of college readiness to inform high school interventions or 
policies. Rather, a range of variables is important to assess postsecondary 
preparation (College Board, 2012; Westrick, 2014). The Community College 
Research Center recommends that high school GPA predicts college success 
more accurately than do exams (Belfield & Crosta, 2012; Scott-Clayton, 2012; 
Scott-Clayton & Stacey, 2015). Similarly, according to Achieve, “Placement tests 
should not be used as a substitute for building more comprehensive measures 
of college and work readiness into the state high school accountability system. 
The majority of the college placement tests reviewed by Achieve are narrowly 
focused on a subset of knowledge and skills” (Achieve, 2007). Given these 
critiques, policymakers should consider using a combination of accountability 
measures when assessing college and career readiness. 


Recommendations for implementing these assessments as part of an accountability 
system include the following: 


Customize admissions tests when incorporating them into statewide testing 
systems. There are gaps in what any assessment will measure. According to 
Achieve, neither the ACT nor the SAT includes the full range of concepts and 
skills reflected in the ADP benchmarks (2007). While this research is dated, 
states may need to consider supplementing admissions tests with additional 
questions or exams to capture the full range of academic standards as high 
school standards evolve. 


Modify high school tests to measure college readiness. As an alternative to 
recommendation #3, high schools may wish to use existing high school exams 
to measure college readiness rather than incorporate college admissions or 
placement exams. In this case, adding targeted questions that align with 
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college curricula may streamline testing systems so that fewer exams are 
necessary (Achieve, 2007; Kurlaender et al., 2016). 


Provide support for districts and schools to assist students who do not meet 
benchmarks. Using college admissions or placement exams in a state 
accountability system should inform numerous discussions about college 
readiness. However, the information gained from these exams is only the first 
step in improving students’ college readiness. Districts, schools, and students 
also need supports such as college remediation courses or transition curricula 
in the K-12 setting to improve preparedness (Barnett, Fay, Bork, & Weiss, 2013; 
Hodara, Smith Jaggers, & Mechur Karp, 2012). 
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